Financial markets analysis has been and remains a topic of intense research interest since the seminal work of Markowitz [1] detailing his theory on portfolio choice, for which he was awarded the Nobel Prize in 1990. The rapid advancements of Machine Learning (ML) and, more specifically those made in the field of Deep Learning (DL) and Deep Reinforcement Learning (DRL), further fueled interest in the field. Financial markets analysts began using ML-based techniques and combining them with their own knowledge of the field [2]. As early as 1992, Neural Networks (NNs) were already being used for equity index futures trading [3]. More recently, DL research in financial market analysis has focused on high frequency trading, i.e., an algorithmic financial trading method where high speeds and large volumes are the main characteristics. The kind of data used in works that focus on this type of trading include Limit Order Book (LOB) data [4] as well as candle data for assets such as FOREX or Cryptocurrencies [5]. Candle data contain the Open, High, Low and Close prices for assets in a requested frequency, e.g., at the minute or hour level. Price forecasting is a first step towards solving the very complex task of portfolio management, and has proved to be a sufficiently difficult problem to tackle itself. One way to sufficiently solve it is by transforming the problem into one of classification, i.e., predicting the price movement instead of its actual value in the next step [4].

Time series forecasting is a crucial component of many important applications, ranging from predicting thebehavior of financial markets [5], to accurate energy load prediction [13]. Even though the large amount of data that can be nowadays collected from these domains provide an unprecedented opportunity for applying powerful deep learning (DL) methods [23, 41, 24], the high-dimensionality, velocity and variety of such data also pose significant and unique challenges that must be carefully addressed for each application. To this end, many methods have been proposed to analyze and forecast time series data. For example, traditional approaches employ adaptive distance metrics, such as Dynamic Time Wrapping [4], to tackle these kind of tasks. However, with the advent of DL the interest is gradually shifting toward using neural network-based methods, including recurrent and convolutional architectures [25, 7], that seem to be more effective for handling such kind of data. It is worth noting that other approaches for time series analysis also exist, such as using the Bag-of-Features model (BoF) [35]. The BoF model was recently adapted toward efficiently processing large amounts of complex and high-dimensional time series [2, 1, 32], due its ability to analyze objects that consist of a varying number of features, as well as withstanding distribution shifts better than competitive methods [29]. The Bag-of-Features model (BoF) involves the following pipeline [35]: a) Several feature vectors are extracted from each input object, e.g., an image or time series. This step is called feature extraction and allows for forming the feature space, where each object is represented as a set of feature vectors.